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DMYTRO IVANENKO 

ASYMPTOTICALLY OPTIMAL ESTIMATOR OF 
THE PARAMETER OF SEMI-LINEAR 
AUTOREGRESSION 

The difference equations = a/(^fe-i) + where (e^) is a square 
integrable difference martingale, and the differential equation d^ = 
—af{^)dt + d?7, where r/ is a square integrable martingale, are con- 
sidered. A family of estimators depending, besides the sample size 
n (or the observation period, if time is continuous) on some random 
Lipschitz functions is constructed. Asymptotic optimality of this 
estimators is investigated. 

1. Introduction 

Discrete time 
We consider the difference equation 

^fc = a/fe-i) fceN, (1) 

where is a prescribed random variable, / is a prescribed nonrandom 
function, a is an unknown scalar parameter and (e^) is a square integrable 
difference martingale with respect to some flow (F^, k G Z+) of cr-algebras 
such that the random variable is Fo-measurable. In the detailed form, 
the assumption about (e^) means that for any k is Ffe-measurable, 

Eel < oo (2) 

and 

E(efc|F,.„i) = 0. (3) 

The word "semi-linear" in the title means that the right-hand sides of 
dl]) depend linearly on a but not on ^. 
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We use the notation: Li. p. - limit in probability; - the weak conver- 
gence of finite-dimensional distributions of random functions, in particular 
convergence in distribution of random variables. 

Let for each /c G Z+ /i^ = hk{oJ,x) be an Ffc_i ® B-measurable function 
(that is the sequence {hk) be predictable) such that 

E[(|efc+i| + \hk{ik)\] + nhk{iK)\ < oo. 

Then from (PP) - (jS]) we have E — af{ik)) hk{^k) = 0, whence 

provided (Ef{^k)hk{C,k)) 7^ 0. This prompts the estimator 

/n.-l \ /n-l \ ^1 

an=[Y^ ^k+ihk{^k) E fi^k)hki^k) , (4) 

\fc=0 / \fc=0 / 

coinciding with the LSE if hk{x) = f{x) for all k. 

Continuous time 

We consider the differential equation 

dm = -afm)dt + dr]{t), teR, (5) 

where r]{t) is a local square integrable martingale w.r.t. a fiow (F(t)) such 
that the random variable ^(0) is F(0)-measurable. 

Let h{t, x) be a predictable random function such that for all t G R+ 

E[(iewi + \afm)\) \Ht,m)\] + nHt,m)\ < oo. 

Let us multiply ([5]) on h{t,C,{t)) and integrate from to T. The same 
rationale as in the discrete case yields the estimator 

= - ( 1^ Ht, atm^ ( fmnt, amt^ , (6) 

coinciding with the LSE if h{t, x) = f{x). 

Asymptotic normality of ^/n (^An — , where An is the LSE of a matrix 
parameter A, was proved in [1] under the assumptions of ergodicity and 
stationarity of {C,n) ■ Convergence in distribution of this normalized deviation 
was proved in [2] with the use of stochastic calculus. Ergodicity and even 
stationarity of (e^) was not assumed in [2], so the limiting distribution could 
be other than normal. 

The goal of the article is to match a sequence (hk) (if time is discrete) or 
a function h{t, ■) (if time is continuous) so that to minimize the value of some 
random functional Vn which, as we shall see in Section 3, is asymptotical 
close in distribution to some numeral characteristic of the estimator (in 
case the latter is asymptotically normal this characteristic coincides with 
the variance). 
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2. The main results 

Discrete time 

Denote al = E[ek|Fk-i], fik = hk{^k)- Let Lip(C) denote the class of 
functions satisfying the Lipschitz condition with some constant C and equal 
to zero at the origin, Lip = lJc>oLip(C'), and let H(C) denote the class 
of all predictable random functions on Z_|_ x R (discrete time) or R_|_ x R 
(continuous time) whose realizations hk{-) (respectively h{t,-)) belong, as 
functions of x, to Lip(C), H = |J(^>oH(C). Predictability means P ® B- 
measurability in {u!,t,x) (the cr-algebra P is defined in [4, p. 28], [6, p. 13]). 

We are seeking for {hj.) G H minimizing the functional 

ly-n-l 2 2 

m,...,/^„-i) = --^^^f^^^. (7) 

Theorem 1. Let 

Vn(ho, ■ ■ ■ ,hn-i) = min Vn{ho, . . . , hn-i). (8) 

ho,...,hn-i€rl 

Then 

n—1 n—1 

^k+ih fi^i)J^i = fi^k) J2 ^i+il^i^ ^ = 0,^-1- (9) 

i=Q 1=0 

Proof. To obtain the necessary conditions for extremum of the functional Vn 
([9]) we will vary [3] just one of functions hk, = 0, — 1, leaving the other 
functions without changes. Thus regarding \4(/io, • • • , hn-i) as a functional 
depending on only one function Vn{ho, . . . , hn-i) = Vnihk). 

Let's choose some scalar function (7 G H and denote g\{x) = hk{x) + 
\{g[x) - hkix)), v{X) = Vnigx). 

Obviously, gx & H so the minimum of v{X) is attained at zero and 
therefore 

v'{0) = 0. (10) 
The expression for the left-hand side is 

2nigi^k) - f^k) {al^iMT:=o fi^^ ' fi^k) Er=o ^liP^j 

Hence in view of (11 01) we obtain the i th equation of system ([9]). 

It remains to apply this argument to each function hk, k = 0,n — 1. 
Remark. The Lipschitz condition was not used in the proof. It will be 
required in Section 3. 

Corollary 1. Let / G Lip(C) and there exist a constant g > such that 
o'k — ^ ^- Then hi{x) = f{x)/af^-^^, i = 0,n — 1, is a solution to the 

problem ([S]). 
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Continuous time 
Let m denote the quadratic characteristic of t]. 

We shall match h = h{u!, t, x) from H(C) (C is independent of t) so that 
to minimize the value of the functional 

Theorem 2. Let 

Vrih) =imnVT{h). (12) 

Then for all g 

/o^ h{t, m)9it, m)dm{t) fmm, amt = 

lo fm)9it, m)dt h{t, e(t))Mm(t). 

(13) 

Proof. Let's choose some scalar function (7 G H and denote gx{t,x) = 
hit,x) + Xg{t,x), v{X) = Vrigx). 

Obviously gx(t, ■) G H so the minimum of v{X) is attained in zero and 
therefore 

v'{0) = 0. (14) 
The expression for the left-hand side is 

v'{o) = 2T {j^fmMt,m)dty'x 

fmMt, e(t))dt r h{t, atmt, ew)dm(t)- 



fm)9it,m)dt / h{t,m'dm{t) 



Hence in view of (1141) we come to ( fT3l) . 

Corollary 2. Let / G Lip(C), m be absolutely continuous w.r.t. the 
Lebesgue measure and there exist a constant g > such that for all t 
m>q. Then h{t,x) = f{x)/m is a solution to the problem (fT2|) . 

3. An illustration 

Denote = E(- ■ ■ |Fo), Qn = I n=l f{ik)^ik, G„ = ^ ELi ^Ul-i- 
We denote E° = E(- • • |Fo) and introduce the conditions 

CPl. For any r G N and any uniformly bounded sequence (a^) of R-valued 

Borel functions on R'' 



1 p 
~ (Q^fc(efc-r+i, ■ • ■ , efc) — E°afe(efe-r+i, • • • , Cfc) ) — ^ 0, 



k=r 
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n— 1 

- X! {'^k'^ki^k-r+i, • • • , Cfe) - E^alak{ek-r+i, • • • , efc)) 0. 

^ A:=r 

CP2. For such r and (afc) the sequences 

( - ^ E°ak{ek-r+i, • • • , efc), n = r + 1, . . . J , 

( - V E°cTfcafc(efc_r+i, . . . , efc), n = r + 1, . . . ] 

converge in probabihty. 

Denote /o(a;) = x and, for r > 1, 

fr{XQ, . . . , Xr) = 0,f{fr-l{XQ, . . . , Xr-l)) + X^. 

Then 

Lemma 1. Let conditions CPl and CP2 6e fulfilled. Suppose 

also that 

1 " 

hm hm-y Ee?/{|eJ > iV} = (15) 

k=l 

and there exist an Y Q-measurahle random variable v such that for all k 

al < V. (16) 
and positive numbers C, Ci such that 

|a|C<l, (17) 
/eLip(C), (/ifc) eH(Ci). Then 

(G'„,g„)^(G,g). (18) 



Proof Denote ^ = /,(0, e,.r+i, ...,ek),^l = hu{Ck). Qn = i Ek=r /(^D/^L 

= ^Efc=r^i(AiLi)^- We claim that conditions ©, ([IS]), 
and the relation 

(Q;,GD^(g^G'^) as n^oo (19) 

imply (fT8|). 

Let Xr denote {xi, . . . , Xr) G R''. Then under the assumptions on / and 
hk for any > 

lim sup \fr{x,Xr) - fr{0,Xr)\ = 0, 
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whence with probabihty 1 for any k 

hm sup \f{fr{x,Xr))hk{fr{x,Xr))-f{fr{0,Xr))hk{fr{0,Xr))\=0, 

(20) 



\x\<N,XreR 



hm sup \h{fr{x,Xr)y - h{fr{0,X,)y\ = 0. 



\x\<N,Xr£K 



These relations was proved in [5]. 

Let us prove that from conditions ([2]), ([3]), (fTSjl . (fT6|l and (fTTjl it foUows 
that almost surely 



lim limE° I Q„-Qn =0, lim lim E°|G„ - = 0. (21) 

r— »oo n— >oo r^oo n^oo 

By (120]) for any iV > 

1 n— 1 

Jiim h^- ^ E|/(e,) ® /ifc - /(eD/^^J^{iefc| < AT} = 0. (22) 

^ k=r 

Denote = I{\ik\ > N}, Jf = I{\ek\ > (1 - C)iV}, 6f = EO|aPxf • Due 
to (fT7|) and because of (hk) G H(Ci) 

E°|/(6)/i.|xf < , 
Hence and from (E]), ([3]), ([ISD-dIT!) we get by Corollary 1 [5] 

-| n— 1 
fc=0 

Further, for k > r, 

EV(eD/ifcl = E°|/(/r(0, e^-r+i, . . . , 6,))||/i,(/.(0, e,„,+i, . . . , e,))|, 
whence ^ 

E|/(a)/^^|xf <CCiE(^X:^^|e,_,|j xf. (24) 

Writing the Cauchy - Bunyakovsky inequality 

/r-l \ 2 r-l r-1 



u=0 / j=0 i=0 



|2 



we get for an arbitrary L > 



E (E[=o' 

(1 - C)-' ( E E CVL,J{|6,„.| > L} + L^P{|a| > AT} E C^M . (25) 



r— 1 r—1 



i=0 i=0 
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In view of ([2]), ([3]) Lemma 1 [5] together with (fT7|l and f|T5|) implies that 

1 

hm Iim-yP{|efc| > A^} = 0. (26) 
Obviously, for arbitrary nonnegative numbers Uq, . . . , Wr-i, ^^i, • • • , Vn-i 

n—1 r—1 r—1 n—1 

k=r i=0 i=0 j=l 

SO conditions (fTTI) and (fTBl) imply that 

-| n—1 r—1 

il- -pS;^ E EE^^4-./{|e.-.| > = 0, 
whence in view of ( |24|) - (1261) 

1 n—1 

hm snp^-j2miau\x^ = 0- 

k=r 

Combining this with fl22l) and fl23l) . we arrive at the first relation of fl2T]) . 
The proof of the second relation of fl?Il) is similar. 
The details can be found in [5]. 

From f|T9|) . and fl2Tl) we obtain that the sequence ((Q^, G""),r G N) con- 
verges in distribution to some limit (Q, G) and relation ([T8|l holds. 
Let us check (fT9|) . Condition CPl implies that 

lim TErE°|Q: - E°Qn = 0, lim TErE°|G: - YPGV\ = 0. 

It remains to note that under condition CP2 for any r G N the sequences 
(E^GJ^) and (E°QJ^) converge in probability. 

By construction Vn{ho, . . . , hn-i) = GnQn"^- The value Q„ = is ex- 
cluded by the choice of the tuple (/iq, • • • , hn-i) minimizing Vn- 
Corollary 3. Let the conditions of Lemma 1 be fulfilled and Q 7^ a.s. 
Then K ^ V, where V = GQ~\ 

Having in mind the use of stochastic analysis, we introduce the processes 
a„(t) = d[nt] and the flows F„(t) = F[nt] with continuous time. 

Theorem 3. Let conditions of Lemma 1 he fulfilled. Then y/n{dn{-) — a) 

where P is a continuous local martingale with quadratic characteristic 

{m) = tV, (27) 

and initial value 0. 
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Proof. Denote Y^{t) = j^Ei Ji e/c/^fc-i- Then because of (jlj) 

^/^(a„(t) - a) = Yr,{t)Q-\ (28) 



By construction and conditions (|2]), ([3]), flTTl) F„ is a locally square inte- 
grable martingale with quadratic characteristic 

It was proved in [5] that under conditions ([2]), ([3]), f|T5l) . f|T6|) . f|T71) and 

( ITSi) yn(Q^n(-) — a) F(-)(5^^, where F is a continuous local martingale 
w.r.t. some flow {F(t),t G R+) such that {Y)(t) = tG and the random 
variable Q is F(0)-measurable (and so does G, which can be seen from the 
expression for (Y)). In view of Lemma 1 it remains to note that Vn = 
(y„)(l)g;2 andV=(F)(l)Q-2. 

Remark. This theorem explains the form of functional In the most 
general case (without conditions CPl and CP2) the denominator (l28ll in 
limit is an F(0)-measurable random variable, and the numerator tends to 
quadratic characteristic at the point t = 1 of the continuous local martingale 
Y. Thus, the numerator ([7]) is the quadratic characteristic at t = 1 of 
the pre-limit martingale Yn, and the denominator satisfies the law of large 
numbers. Minimizing pre-limit variance in (hk) G H(Ci), we lessen the 
value of limited variance of the normalized deviation of estimator (jl]) . 

Let further h^i^x) = f{x)/a1j^^. Recall that {hk, k = 0,n — 1) is a 
solution to the problem ([8]). For such we have 

Corollary 4. Let the conditions of Corollary 1 and Theorem 3 be fulfilled. 
Then 

V= ( liml.i.p.-VE°^^^^ . 

Proof. Obviously Vn = Qn^- By Lemma 1 Q„ Q, where Q = lim l.i.p. E^Q^. 
To complete the proof it remains to note that QL^ = - J^^Zl E^ '^^f''^ . 



4. An example 



Suppose that / G Lip(C), h^. G H(Ci) condition f|T71) be fulfilled. Let 
also e„ = 'jnbniC.n-i) , where (7„) be a sequence of independent random 
variables with zero mean and variances <^^, |7a;| < C2, bn G H(C3) and 
C + C2C3 < 1. Let also E^^ < ^ 

For Ffc we take the a-algebra generated by ^o! 7i; • • • ; 7fc- 
Then al = 4hk{ik-i? and (e„) satisfies (0), ([3]). 
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Denote further 

fr{Xo, ...,Xr)= af{fr-i{Xo, Xr-l)) + XrKifr-liXo, • • • , Xr-l)), 

-| n— 1 

k=r 

Gn = ^Ylk=l'^k+i^k+i{^k)^{p'kY- Similarly to the proof of Lemma 1 we 
obtain 

lim Ih^E^lGn -Gl\ = 0, lim lh^E^\Q„ - Qll = 0. 

r—*oon-*oo r— >co n^oo 

Items in G^^ and depends on 7^.^+1, . . . , 7^ then they satisfy the law 
of large numbers in Bernstein's form. 

If besides e„ satisfies CP2 and Q ^ then Theorem 3 asserts (1271) . If 
herein i(^^\2 G Lip then hk(x) = ^{^^^2 is a solution to the problem ([8]) 
and ^ ^ 

V = lim i.i.p. y E°— . 

Example. Let bn = b, hn = h and 7^ be i.i.d. random variables. In view of 
expressions for and GJ^ we may confine ourselves with the case ak = a. 

By the Stone - Weierstrass theorem for a-compact spaces [7, p. 317] 
a can be uniformly on compacta approximated with finite linear combina- 
tions of functions of the kind gi{xi) . . . gr{xr)- By the choice of and the 
assumptions on (7^) 

r 

E°^l(7/fc-r+l) • • • ^r(7fc) = n ^9i{ll)- 

i=l 

Hence and from the above assumption on (7/^) condition CP2 emerges. 

Acknowledgement. The author is grateful to A. Yurachkivsky for helpful 
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